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Sir: 

This brief is in furtherance of the Notice of Appeal filed December 11, 
2008, and is submitted responsive to the Final Office Action issued July 24, 2008. 
The filing of this Appeal Brief is timely, the time for response being June 1 1 , 2009 
with the concurrent filing of a four-month Petition for Extension of Time and the 
appropriate fee. Accordingly, appellant submits the following: 
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I. REAL PARTIES IN INTEREST 

The real party in interest in this mater is the sole inventor, Lester F. Ludwig 
(hereinafter "Appellant"). 
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II. RELATED APPEALS AND INTERFERENCES 

Currently, there are ten other related appeals which have been filed. 
These appeals have been filed in the following applications: 



Docket No. 


App. Ser. No.: 


App. filing date: 


Appeal filed: 


2152-3005 


09/812,400 


March 19, 2001 


January 14, 2008 


2152-3014 


10/676,926 


September 30, 
s>nm 


March 31 , 2008 


2152-3018 


10/676,249 


September 30, 
2003 


April 2, 2009 


2152-3020 


10/683,915 


October 10, 2003 


December 1 1 , 2008 


2152-3023 


10/680,591 


October 6, 2003 


January 31, 2008 


2152-3026 


10/703,023 


November 1 5, 2003 


February 16, 2007 


2152-3027 


10/702,262 


November 5, 2003 


October 1, 2007 


2152-3030 


10/702,415 


November 6, 2003 


August 8, 2008 


2152-3043 


11/004,746 


December 3, 2004 


April 2,2009 


2152-3044 


11/040,163 


January 21 , 2005 


May 11,2009 



Appellant notes that there are four additional pending applications 
containing substantially the same disclosure as the above-identified applications, 
and which are assigned to the same Examiner as the present application and the 
above-mentioned applications. Appellant anticipates that each of the four 
pending applications, if rejected under similar circumstances as the eleven cases 
in appeal, may unfortunately also require an appeal to the Board of Appeals and 
Interferences in order to obtain fair examination. Appellant will endeavor to 



-4- 



Attorney Docket No. 2152-3020 



update this section of the present Appeal Brief when necessary to reflect the 
current status of such related appeals. 
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III. STATUS OF CLAIMS 



Claims 1-118 are all the claims in the application, claims 1-24 and 63-118 
have been withdrawn responsive to a Restriction Requirement. Claims 25-62 
remain pending. Claims 25-27, 29-35,37,39-46,48-54, and 62 stand rejected 
under 35 U.S.C. §1 02(b) as being anticipated Kimpara et al. (US 5,310,962). 
Claims 28,36,38,47, and 55-61 stand rejected under 35 U.S.C. §1 03(a) as being 
obvious over Kimpara in view of Toriurni (US 6,062,868). Claims 25 and 44 are 
the independent claims. A copy of the claims appears in the Appendix of Claims 
on Appeal attached to the Appeal Brief. 
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IV. STATUS OF AMENDMENTS 



Claims 1-118 were filed on October 10, 2003, and claims 1-24 and 63-118 
were withdrawn on August 15, 2007 in response to the Restriction Requirement 
of May 17, 2007. Claims 25, 31-36, 38-42, 44, 50-55, and 57-61 were amended 
on March 28, 2008 in the response to the non-final Office Action of December 28, 
2007. No amendment has been filed subsequent to the Final Office Action 
issued July 24, 2008. 
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V. 



SUMMARY OF CLAIMED SUBJECT MATTER 



The invention provides for the integration of video into electronic music 
technology. Cameras mounted on instruments, such as guitars, provide displays 
during performance, recording, and for creating control signals. Control signals 
may be extracted from images captured from cameras. The outgoing control 
signals are MIDI control signals, and extraction algorithms may be selected and 
controlled by MIDI signals. A range of video signal generation, video signal 
processing, and camera image capture control functions relevant to the use of 
video in live performances may be controlled by MIDI signals to unify a common 
control infrastructure for a live performance environment. 

Independent claim 25 describes a system comprising a first image 
processing element and a processor for extracting and generating control signals 
from at least one video signal. 

The first image processing element receives a first incoming video signal 
and detects at least one image-related component within an image represented 
by said first incoming video signal. The processor generates at least one 
outgoing control signal from the image-related component. The outgoing control 
signals comprise at least one of a MIDI continuous controller message and a 
MIDI note message. 

Paragraphs [0313]-[0314] describe the high-level features and advantages 
of claim 25 regarding the control elements and controllers because image 
processing, image recognition, and image motion tracking utilities can be applied 
to video signals. In claim 25, such image processing, image recognition, and 
image motion tracking functions are provided by image processing element and 
processor. 

Paragraphs [0321]-[0331] further describe types of image processing, 
image recognition, and image motion tracking functions that may be performed 
by the image processing element and processor of claim 25 including brightness 



-8- 



Attorney Docket No. 2152-3020 



and color detection, and averaging. Example usage and control signals are 
shown. 

Various exemplary analog, digital, and processor embodiments are 
discussed, specifically the use of a MIDI interface. The specification provides 
examples these capabilities and how they may be used to trigger events and 
continuously control sound, light, and special effects. 

Examples of the recognition of human hand posture, position, and 
proximity to the camera in 3-space are provided. Simple hand orientation, 
posture geometry, facial expression, lip motion, imaged characters, and imaged 
patterns may be used to create specific control signals, and dynamic gestures of 
the hand, arm, body, etc. may be recognized and used to generate control 
signals. The use of multiple cameras to expand these capabilities in include 
additional responsiveness to the hand position, posture, and motion in three- 
dimensional space 

Paragraphs [0464-0465] and FIG. 47 describe and illustrate the features 
of claim 25 as applied to detecting patterns in non-equilibrium chemical reactions 
and to visually monitor the patterns to derive a multitude of parameters to 
generate control signals. These control signals may be used to control any one 
or more of note events, timbre modulation, lighting, and special effects. In 
addition, Incoming control signals provided by or extracted from video cameras 
or body position indicators (gestures, dance, and stage positions) can be used to 
control the evolution and influence the shapes of the chemical patterns. 

Independent claim 44 is a companion to claim 25 and describes a method 
for extracting and generating control signals from at least one video signal. The 
method comprises receiving a first incoming video signal at a first image 
processing element, wherein said first image processing element detects posture 
of a human hand in an image represented by said first incoming video signal, and 
processing the detected posture in the image, resulting in an outgoing control 
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signal comprising at least one of a of MIDI format continuous controller message 
and a MIDI note message, the outgoing control 

Independent claim 44 relies upon the same portions of the specification 
described for independent claim 25. 
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VI. 



GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



Claims 25-27, 29-35,37,39-46,48-54, and 62 stand rejected under 35 
U.S.C. §1 02(b) as being anticipated Kimpara et al. (US 5,310,962). Claims 
28,36,38,47, and 55-61 stand rejected under 35 U.S.C. §1 03(a) as being obvious 
over Kimpara in view of Toriurni (US 6,062,868). 
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VII. ARGUMENT 



A. Prosecution History 

1. Restriction Requirement 

The application was originally filed with claims 1-118. On May 17, 2009, 
the Examiner issued a restriction requirement requiring the appellant to elect one 
of four groups of claims. In response, on August 15, 2007, the appellant elected 
to prosecute claims 25-62 drawn to extracting and generating control signals 
from video signals. 

2. Non-final Office Action 

In the first non-final Office Action dated December 28, 2007, the Examiner 
rejected independent claims 25 and 44 as being anticipated by Kimpara et al. 
(US 5,310,962) stating that Kimpara teaches all the features of the independent 
claims including "processing results in an associated at least one outgoing 
control signal comprising a signal of MIDI format" in FIG 1. 

In response, on March 28, 2008, the appellant amended the independent 
claims to recite "to generate at least one outgoing control signal comprising at 
least one of a MIDI continuous controller message and a MIDI note message." In 
discussing how the amended independent claims overcome the Kimpara 
reference, the appellant argued that Kimpara does not teach that MIDI 
continuous controller message and a MIDI note message are generated in 
response to the image processing, and further that Kimpara's control signal is 
limited to a tempo control signal. 

Claim 44 is further amended to recite the additional limitation of "the 
outgoing control signal being generated responsive to the detected posture of the 
human hand." The appellant argues that this feature is not taught by Kimpara. 

2. Final Office Action and Advisory Action 
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In response, the Examiner issued the Final Office Action on July 24, 2008 
in which he maintained the previous rejections stating "The amendments provide 
little change to the previously submitted claims. MIDI signals are used to control 
sound output. The sound is inherently based on a note message which controls 
the sound output, wherein in different sounds are base on different notes. The 
claims remain rejected." See, Final Office Action, p. 6. 

The appellant responded on October 24, 2008 stating that the Examiner's 
rejections did not address the amendments made in response to the non-final 
Office Action. The appellant stated that the Examiner's reasons for the rejections 
referenced features now deleted ("decomposition element", claims 25 and 44) 
and failed to address newly added features ("MIDI continuous controller 
message" and "MIDI note message", claims 25 and 44; "the detected posture of 
the human hand", claim 44). 

The appellant further objected to the use of inherency for rejecting the 
independent claims. The Examiner stated that "Kimpara et al. inherently 
discloses at least one outgoing control signal comprising a signal of MIDI format 
at least one of a MIDI continuous controller message and a MIDI note message" 
(see, Final Office Action, p. 2) without providing any reasoning to support the 
determination that the claimed MIDI features, which are allegedly inherent 
characteristics, necessarily flow from the teachings of Kimpara. 

In argument, the appellant provided a summary of MIDI messages by the 
MIDI Manufacturers Association, an explanation of MIDI timing concepts, and a 
list of MIDI controller commands. These establish that the MIDI tempo data of 
Kimpara is entirely different than a MIDI continuous controller message or a MIDI 
note message. Consequently, the MIDI tempo data output of Kimpara cannot 
therefore teach or suggest "at least one of a MIDI continuous controller message 
and a MIDI note message," as required by claims 25 and 44. 



- 13- 



Attorney Docket No. 21 52-3020 



In the Advisory Action issued on December 2, 2008, the Examiner stated 
"the claims are met by the prior art. The applicant's arguments are not 
convincing. The response will not be entered." 

In response, the appellant filed a Notice of Appeal on December 11, 2008. 

B. The Examiner Has Not Shown That Kimpara Anticipates The 
Independent Claims 

Independent claims 25 and 44 recite "at least one outgoing control signal 
comprising at least one of a MIDI continuous controller message and a MIDI note 
message." Kimpara teaches only that tempo data is transmitted to a MIDI device. 
See, Kimpara, col. 4: 27-47 ("In the step S6, a tempo control signal or its data are 
generated based on 35 data of above time difference in the step S6. ... the 
tempo data of MIDI (Musical Instrument Digital Interface) standard are to be 
outputted to MIDI device. Meanwhile, it is possible to use this 45 performance 
tempo control apparatus as a tempo generator of automatic performance 
apparatus by outputting the tempo clock itself.") 

a. Tempo data differs from the MIDI note message. Table 1 of 
the Summary of MIDI Messages (MIDI Manufacturers Association), Appendix A of 
appellant's response to the Final Office Action, shows the MIDI Note On and 
MIDI Note Off messages: 

Table it MIDI 1*0 Spec iticst ion Keasage Suimary 
Updated 19DS By the HZDl Manufacture re Association 

Status Data Byte M Deecription 

D7 - - - -DO D7-— DO 

Channel Voice Messages [rirtnn « 0-15 (MIDI Channel Number 

10 OOnnnn OKkkkk&k note off event . 

$vvvvwy This message is sent when a 

note is released (ended) . 
(kfefeJOcfck) is the key I note) nuxaber. 
Cvwvwv) is the velocity. 

lOOlnnoa dkfckkkkfc. Nate On eve&t . 

Owwwv This sues sage is cent when a 

note is depressed (starts , 
(kkkhkkk) is the key (note) number, 
{vr/vvw) is the velocity. 
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The same Table 1 shows the Timing Clock message: 




Timing Clock. 

Sent 24 tines per quarter note when 
synchronization is re<juired (see text) . 



As is shown in Table 1, the Timing Clock message is not the same as an 
MIDI Note message, and Kimpara's tempo control signal cannot teach claim 25's 
and claim 44's "at least one outgoing control signal comprising at least one of ... 
a MIDI note message." 

b. Tempo data differs from the MIDI continuous controller message. 
"Control Change" messages (frequently called Continuous Controller) are quite 
versatile; they are usually generated by a musician using knobs, sliders, 
footswitches, or pressure on a physical MIDI controller (or MIDI-equipped 
instrument). While the response to these messages is generally totally up to the 
receiving device, they are typically used to change the tone, timbre, or volume of 
an instrument's sound. 1 

Table 1 of the Summary of MIDI Messages (MIDI Manufacturers 
Association), Appendix A of appellant's response to the Final Office Action, shows 
the format of the MIDI continuous controller message: 



lOllsnnn 



occccccc 
Owwvw 



Control Change. 

This mes&age. £o sent fctoen & controller 
value changes. Control. lore include devices 
mieh a& pedals and lever©* 
Controller msaber© 120-127 are reserved 
a« * Channel Kode Messages* (below) . 
(ccccccc) is the controller number (0-H9J . 
(wvww) is chc controller value {0*127) . 



Extracted from Wikipedia article "MID1 1 .0", accessed at 
http://en.wikipedia.org/wiki/The_MIDI_1 .0_Protocol on June 9, 2009. 
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The field "nnnn" indicates the MIDI channel number wherein each channel may 
have up to 128 controllers. 

MIDI Controllers can be divided into two main types: those that are 
variable or continuous (like a knob) and those that are either on or off (like a 
switch). The architects of MIDI have reserved Controllers 122 and upwards for 
selecting the various MIDI modes (Local On/Off, All Notes Off, Omni Off, Omni 
On, Mono On and Poly On). The remaining slots are broken down into 
Continuous Controllers 0 to 63, Switch Controllers 64 to 95 and Undefined 
Controllers (free for future use) 96 to 121. MIDI Controllers 0 to 31 are 
duplicated in slots 32 to 63 for the purpose of generating information with a much 
higher resolution than 128 steps. Controllers 0 to 31 handle the most significant 
'bit' (MSB) of the data while Controllers 32 to 63 handle the least significant bit 
(LSB). 2 

Many MIDI devices use a defined set of controllers such as that presented 
in Appendix C of appellant's response to the Final Office Action. All these 
controllers relate to the tone, timbre, or volume of an instrument's sound, but not 
to the tempo control of a musical piece. 

For at least this reason, Kimpara's tempo control signal cannot teach claim 
25's and claim 44's "at least one outgoing control signal comprising at least one 
of a MIDI continuous controller message." 

C. The Examiner Has Failed To Show That Kimpara's Tempo Control 
Signal Inherently Teaches A MIDI Continuous Controller Message Or 
A MIDI Note Message 

The Examiner has relied upon inherency to reject independent claims 25 
and 44 without providing any reasoning to support the claimed inherency. In 



2 Extracted from "Sound on Sound, MIDI BASICS: Part 3" web page, 
accessed at http://www. soundonsound. com/sos/1 995_articles/oct95/midibasics3. 
html on June 9, 2009. 
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rejecting the independent claims, the Examiner made the following statement: 
"Kimpara et al. inherently discloses at least one outgoing control signal 
comprising a signal of MIDI format at least one of a MIDI continuous controller 
message and a MIDI note message, the outgoing control signal being generated 
responsive to content in the image represented by said first incoming video 
signal." See, Final Office Action, p.2. The Examiner fails to explain why 
Kimpara's tempo control signal inherently encompasses the either one or both of 
a MIDI continuous controller message and a MIDI note message. 

The Examiner, when rejecting a claim based on inherency, is required to 
"provide a basis in fact and/or technical reasoning to reasonably support the 
determination that the allegedly Inherent characteristic necessarily flows from the 
teachings of the applied prior art." See, MPEP 2112, citing Ex parte Levy, 17 
USPQ2d 1461, 1464 (Bd. Pat. App. & Inter. 1990). Here, the Examiner made the 
bold and incorrect statement that it is inherent that Kimpara's tempo control 
signal includes a continuous controller message and a MIDI note message. 

As discussed supra, each of the tempo control signal (Timing Clock 
message), the MIDI note message, and the MIDI continuous controller message 
is a separate message and the presence of one of these messages does not 
imply the presence of one of the other messages. Therefore, it is not inherent 
that Kimpara's tempo control signal teaches one or both of a MIDI note message 
and a MIDI continuous controller message. The Examiner's statement of 
inherency is incorrect. 

D. Examiner Does Not Address The Substance Of Appellant's 
Arguments Made During Prosecution 

In response to the non-final Office Action, the appellant amended 
independent claims to recite "to generate at least one outgoing control signal 
comprising at least one of a MIDI continuous controller message and a MIDI note 
message." In the Final Office Action, the Examiner merely repeated his 
statement of rejection from the non-final Office Action ("wherein said processing 
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results in an associated at least one outgoing control signal comprising a signal 
of MIDI format (figure 1)." This is the recitation that was amended. In other 
words, the Examiner did not address the amendment. 

However, the Examiner stated that "Kimpara et al. inherently discloses at 
least one outgoing control signal comprising a signal of MIDI format at least one 
of a MIDI continuous controller message and a MIDI note message" without 
further explanation of his reasoning why Kimpara's tempo control signal 
inherently included at least one of a MIDI continuous controller message and a 
MIDI note message. 

In response to the Final Office Action, the appellant provided the Examiner 
with sufficient reasoning to demonstrate that Kimpara's tempo control signal does 
not teach a MIDI continuous controller message and a MIDI note message, 
inherently or otherwise. In response to the appellant's reply, in the Advisory 
Action, the Examiner just stated "the claims are met by the prior art. The 
applicant's arguments are not convincing. The response will not be entered." No 
reasoning was ever presented. 

MPEP § 707.07(f) requires the Examiner to address the substance of the 
appellant's arguments: 

"In order to provide a complete application file history and to 
enhance the clarity of the prosecution history record, an examiner must 
provide clear explanations of all actions taken by the examiner during 
prosecution of an application. 

Where the requirements are traversed, or suspension thereof 
requested, the examiner should make proper reference thereto in his or 
her action on the amendment. 

Where the appellant traverses any rejection, the examiner should, if 
he or she repeats the rejection, take note of the appellant's argument and 
answer the substance of it." 
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The Examiner has never addressed or answered the appellant's 
arguments that Kimpara's tempo control signal does not teach all the limitations 
of the independent claims. 

E. Claims Currently In Condition For Allowance 

As set forth in MPEP 2131, to anticipate a claim, the reference must teach 
every element of the claim. Since, as discussed above, every element of 
independent claims 25 and 44 is not taught by Kimpara, appellant submits that 
the Examiner has not shown these claims are anticipated by Kimpara. 

Appellant has demonstrated a number of errors in the rejections of 
independent claims 25 and 44 and that claims 25 and 44 are patentable. 
Dependent claims 26-43 and 46-62 are patentable as well by virtue of 
dependence on a patentable independent claim. Appellant therefore submits 
that the identified rejections are improper and that the identified claims are 
allowable over the asserted references. Appellant respectfully requests that the 
Board of Patent Appeals and Interferences reverse the decision rejecting the 
identified claims and direct the Examiner to pass the case to issue. 

Respectfully submitted, 



Date: June 1 1 , 2009 By: /C.W. Schmover/ 

Craig W. Schmoyer 
Registration No. 51,007 
Attorney for Appellant(s) 

Customer No. 035884 
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CLAIMS APPENDIX 



1 . (WITHDRAWN) A system for capturing real-time video images from a 
musical instrument, said system comprising: 

a musical instrument comprising an instrument body and a performance 
area, wherein said performance area defines an area of said musical instrument 
where a user physically interacts with said musical instrument; 

a video camera attached to said instrument body to capture real-time 
video images of at least a portion of said musical instrument, wherein said video 
camera generates a video signal representing said real-time video images; and 

a video output interface providing said video signal to an external system. 

2. (WITHDRAWN) The system according to claim 1 , wherein said 
external system comprises a video display visible to an audience. 

3. (WITHDRAWN) The system according to claim 1 , wherein said video 
signal is processed by at least one real-time image processor, wherein said at 
least one real-time image processor is configured to generate an outgoing control 
signal responsive to said video signal. 

4. (WITHDRAWN) The system according to claim 3, wherein said 
outgoing control signal comprises a signal of MIDI format. 

5. (WITHDRAWN) The system according to claim 3, wherein said video 
signal is processed by at least one real-time image processor to generate a 
modified video signal, wherein said at least one real-time image processor is 
controlled by an incoming control signal. 

6. (WITHDRAWN) The system according to claim 5, wherein said 
musical instrument generates said incoming control signal in response to said 
user interaction with said musical instrument. 
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7. (WITHDRAWN) The system according to claim 5, wherein said 
incoming control signal is generated by a physical controller in response to user 
interaction with said physical controller. 

8. (WITHDRAWN) The system according to claim 5, wherein said 
incoming control signal comprises a signal of MIDI format. 

9. (WITHDRAWN) The system according to claim 1 , wherein said video 
signal is recorded by a video recording system. 

1 0. (WITHDRAWN) The system according to claim 1 , wherein said video 
camera is attached to said instrument body in such a location to capture said 
real-time video images from said performance area of said musical instrument. 

1 1 . (WITHDRAWN) The system according to claim 1 0, wherein said 
musical instrument comprises a string guitar, and wherein said performance area 
includes a bridge area of said string guitar. 

12. 12. (Withdrawn) The system according to claim 10, wherein said musical 
instrument comprises a string guitar, and wherein said performance area includes 
a non-string area of said string guitar. 

1 3. (WITHDRAWN) A method for capturing real-time video images from a 
musical instrument, said method comprising: 

providing a musical instrument comprising an instrument body and a 
performance area, wherein said performance area defines an area of said 
musical instrument where a user physically interacts with said musical 
instrument; 

capturing real-time video images of at least a portion of said musical 
instrument using a video camera attached to said instrument body, wherein said 
video camera generates a video signal representing said real-time video images; 
and 

providing said video signal to an external system. 
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14. (WITHDRAWN) The method according to claim 13, wherein said 
external system comprises a video display visible to an audience. 

15. (WITHDRAWN) The method according to claim 13, wherein said 
video signal is processed by at least one real-time image processor, wherein said 
at least one real-time image processor is configured to generate an outgoing 
control signal responsive to said video signal. 

16. (WITHDRAWN) The method according to claim 15, wherein said 
outgoing control signal comprises a signal of MIDI format. 

17. (WITHDRAWN) The method according to claim 15, wherein said 
video signal is processed by at least one real-time image processor to generate a 
modified video signal, wherein said at least one real-time image processor is 
controlled by an incoming control signal. 

18. (WITHDRAWN) The method according to claim 17, wherein said 
musical instrument generates said incoming control signal in response to said 
user interaction with said musical instrument. 

19. (WITHDRAWN) The method according to claim 17, wherein said 
incoming control signal is generated by a physical controller in response to user 
interaction with said physical controller. 

20. (WITHDRAWN) The method according to claim 17, wherein said 
incoming control signal comprises a signal of MIDI format. \ 

21. (WITHDRAWN) The method according to claim 13, wherein said 
video signal is recorded by a video recording system. 

22. (WITHDRAWN) The method according to claim 1 3, wherein said 
video camera is attached to said instrument body in such a location to capture 
said real-time video images from said performance area of said musical 
instrument. 
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23. (WITHDRAWN) The method according to claim 22, wherein said 
musical instrument comprises a string guitar, and wherein said performance area 
includes a bridge area of said string guitar. 



24. (WITHDRAWN) The method according to claim 22, wherein said 
musical instrument comprises a string guitar, and wherein said performance area 
includes a non-string area of said string guitar. 

25. (PREVIOUSLY PRESENTED) A system for extracting and generating 
control signals from at least one video signal said system comprising: 

a first image processing element for receiving a first incoming video signal, 
wherein said first image processing element detects at least one image-related 
component within an image represented by said first incoming video signal; and 

a processor for processing said at least one image-related component to 
generate at least one outgoing control signal comprising a at least one of a MIDI 
continuous controller message and a MIDI note message, the outgoing control 
signal being generated responsive to content in the image represented by said 
first incoming video signal. 

26. (ORIGINAL) The system according to claim 25, wherein said first 
incoming video signal originates from a video camera. 

27. (ORIGINAL) The system according to claim 26, wherein said video 
camera is attached to a musical instrument. 

28. (ORIGINAL) The system according to claim 26, wherein said incoming 
video signal is communicated to a video display visible to an audience. 

29. (PREVIOUSLY PRESENTED) The system according to claim 25, 
wherein an incoming control signal is used by said processor to generate said at 
least one outgoing control signal. 

30. (ORIGINAL) The system according to claim 29, wherein said incoming 
control signal comprises signals of MIDI format. 
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31 . (PREVIOUSLY PRESENTED) The system according to claim 25, 
wherein said processor performs the processing response to a control signal 
extraction algorithm selected from a plurality of pre-stored control signal 
extraction algorithms. 

32. (PREVIOUSLY PRESENTED) The system according to claim 25, 
wherein said processor is configured to recognize at least one pattern in said 
incoming video signal, wherein said at least one pattern is used to generate said 
at least one outgoing control signal. 

33. (PREVIOUSLY PRESENTED) The system according to claim 32, 
wherein said processor is further configured to calculate at least one pattern 
parameter associated with said at least one pattern in said incoming video signal, 
wherein said pattern parameter is used to generate said at least one outgoing 
control signal. 

34. (PREVIOUSLY PRESENTED) The system according to claim 32, 
wherein said processor is further configured to recognize at least one gesture in 
said incoming video signal, wherein said recognition is used to generate said at 
least one outgoing control signal. 

35. (PREVIOUSLY PRESENTED) The system according to claim 34, 
wherein said processor is further configured to calculate at least one gesture 
parameter associated with said at least one gesture recognized in said video 
input signal, wherein said gesture parameter is used to generate said at least 
one outgoing control signal. 

36. (PREVIOUSLY PRESENTED) The system according to claim 25, said 
system further comprising: 

a second video signal image processing element for receiving a second 
incoming video signal, wherein said second video signal image processing 
element detects at least one image-related component from said second 
incoming video signal, wherein 
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said processor is further configured to processes said at least one image- 
related component of said second incoming video signal, wherein said 
processing of said at least one image-related component of said first incoming 
video signal and said at least one image-related component of said second 
incoming video signal results in said at least one outgoing control signal. 

37. (ORIGINAL) The system according to claim 36, wherein said second 

incoming video signal originates from a video camera. 
« 

38. (PREVIOUSLY PRESENTED) The system according to claim 25, said 
system further comprising: 

a second video signal image processing element for receiving a second 
incoming video signal, wherein said second video signal image processing 
element detects at least one image-related component from said second 
incoming video signal, wherein 

said processor is further configured to processes said at least one image- 
related component of said second incoming video signal resulting in a secondary 
at least one outgoing control signal comprising least one of a MIDI continuous 
controller message and a MIDI note message. 

39. (PREVIOUSLY PRESENTED) The system according to claim 36, 
wherein said processor is further configured to recognize at least one pattern in 
said second incoming video signal, wherein said at least one pattern is used to 
generate said at least one outgoing control signal. 

40. (PREVIOUSLY PRESENTED) The system according to claim 39, 
wherein said processor is further configured to calculate at least one pattern 
parameter associated with said at least one pattern in said second incoming 
video signal, wherein said pattern parameter is used to generate said at least one 
outgoing control signal. 
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41 . (PREVIOUSLY PRESENTED) The system according to claim 39, 
wherein said processor is further configured to recognize at least one gesture in 
said second incoming video signal to generate said at least one outgoing control 
signal. 

42. (PREVIOUSLY PRESENTED) The system according to claim 41 , 
wherein said processor is further configured to calculate at least one gesture 
parameter associated with said at least one gesture recognized in said video 
input signal, wherein said gesture' parameter is used to generate said at least 
one outgoing control signal. 

43. (ORIGINAL) The system according to claim 25, wherein said first 
incoming video signal is processed by a real-time image processor to generate a 
modified video signal, wherein said real-time image processor is controlled by an 
incoming control signal. 

44. (PREVIOUSLY PRESENTED) A method for extracting and generating 
control signals from at least one video signal said method comprising: 

receiving a first incoming video signal at a first image processing element, 
wherein said first image processing element detects posture of a human hand in 
an image represented by said first incoming video signal; and 

processing the detected posture in the image, resulting in an outgoing 
control signal comprising at least one of a of MIDI format continuous controller 
message and a MIDI note message, the outgoing control signal being generated 
responsive to the detected posture of the human hand. 

45. (ORIGINAL) The method according to claim 44, wherein said first 
incoming video signal originates from a video camera. 

46. (ORIGINAL) The method according to claim 45, wherein said video 
camera is attached to a musical instrument. 

47. (ORIGINAL) The method according to claim 45, wherein said incoming 
video signal is communicated to a video display visible to an audience. 
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48. (PREVIOUSLY PRESENTED) The method according to claim 44, 
wherein an incoming control signal is used by said processor to generate said at 
least one outgoing control signal. 

49. (ORIGINAL) The method according to claim 48, wherein said incoming 
control signal comprises signals of MIDI format. 

50. (PREVIOUSLY PRESENTED) The method according to claim 44, 
wherein said processorperforms the processing response to a control signal 
extraction algorithm selected from a plurality of pre-stored control signal 
extraction algorithms. 

51 . (PREVIOUSLY PRESENTED) The method according to claim 44, 
wherein said processor is configured to recognize at least one pattern in said 
incoming video signal, wherein said at least one pattern is used to generate said 
at least one outgoing control signal. 

52. (PREVIOUSLY PRESENTED) The method according to claim 51 , 
wherein said processor is further configured to calculate at least one pattern 
parameter associated with said at least one pattern in said incoming video signal, 
wherein said pattern parameter is used to generate said at least one outgoing 
control signal. 

53. (PREVIOUSLY PRESENTED) The method according to claim 5 1 , 
wherein said processor is further configured to recognize a t least one gesture in 
said incoming video signal, wherein said recognition is used to generate said at 
least one outgoing control signal. 

54. (PREVIOUSLY PRESENTED) The method according to claim 53, 
wherein said processor is further configured to calculate at least one gesture 
parameter associated with said at least one gesture recognized in said video 
input signal, wherein said gesture parameter is used to generate said at least 
one outgoing control signal. 
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55. (PREVIOUSLY PRESENTED) The method according to claim 44, said 
method further comprising: 

receiving a second incoming video signal at a second video signal imag- e 
processing element wherein said second video signal decomposition element 
detects at least one image-related component from said second incoming video 
signal; and 

processing said at least one image-related component of said second 
incoming video signal, wherein said processing of said at least one jmage-related 
component of said first incoming video signal and said at least one image-related 
component of said second incoming video signal results in said at least one 
outgoing control signal. 

56. (ORIGINAL) The method according to claim 55, wherein said second 
incoming video signal originates from a video camera. 

57. (PREVIOUSLY PRESENTED) The method according to claim 44, said 
method further comprising: 

receiving a second incoming video signal at a second video signal image 
processing element, wherein said second video signal image processing element 
detects at least one image-related component from said second incoming video 
signal; and 

processing said at least one image-related component of said second 
incoming video signal resulting in a secondary at least one outgoing control 
signal comprising at least one of a MIDI continuous controller message and a 
MIDI note message. 

58. (PREVIOUSLY PRESENTED) The method according to claim 55, 
wherein said processor is further configured to recognize at least one pattern in 
said second incoming video signal, wherein said at least one pattern is used to 
generate said at least one outgoing control signal. 
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59. (PREVIOUSLY PRESENTED) The method according to claim 58, 
wherein said processor is further configured to calculate at least one pattern 
parameter associated with said at least one pattern in said second incoming 
video signal, wherein said pattern parameter is used to generate said at least one 
outgoing control signal. 

60. (PREVIOUSLY PRESENTED) The method according to claim 58, 
wherein said processor is further configured to calculate at least one gesture in 
said second incoming video signal to generate said at least one outgoing control 
signal. 

61 . (PREVIOUSLY PRESENTED) The method according to claim 60, 
wherein said processor is further configured to calculate least one gesture 
parameter associated with said at least one gesture recognized in said video 
input signal, wherein said gesture parameter is used to generate said at least 
one outgoing control signal. 

62. (ORIGINAL) The method according to claim 44, wherein said first 
incoming video signal is processed by a real-time image processor to generate a 
modified video signal, wherein said real-time image processor is controlled by an 
incoming control signal. 

63. (WITHDRAWN) A real-time video signal generation system, said 
system comprising: 

an input interface for receiving incoming MIDI control signals; 

a memory unit for storing previously generated video data comprising real- 
time video images; and 

a processor for generating a video output signal comprising selectable 
portions of said previously generated video data, wherein said video output 
signal is generated according to a video generation algorithm that associates 
said incoming MIDI control signals with particular real-time video images of said 
stored video data. 
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64. (WITHDRAWN) The system according to claim 63, wherein said video 
generation algorithm is selected from a plurality of pre-programmed video 
generation algorithms. 

65. (WITHDRAWN) The system according to claim 64, wherein an 
incoming control signal of MIDI format is used to select a particular video 
generation algorithm from said plurality of pre-programmed video generation 
algorithms. 

66. (WITHDRAWN) The system according to claim 63, wherein said real- 
time video images comprise video stills. 

67. (WITHDRAWN) The system according to claim 63, wherein said real- 
time video images comprise video clips. 

68. (WITHDRAWN) The system according to claim 63, wherein said real- 
time video images comprise graphics animations. 

69. (WITHDRAWN) The system according to claim 63, wherein said real- 
time video images comprise a graphic rendering created from a numerical 
dynamic simulation. 

70. (WITHDRAWN) The system according to claim 63, wherein at least 
one graphics operation is introduced into said video output signal in response to 
said incoming MIDI control signals. 

71 . (WITHDRAWN) The system according to claim 70, wherein said 
graphics operation comprises a raster graphics operation. 

72. (WITHDRAWN) The system according to claim 70, wherein said 
graphics operation comprises a vector graphics operation. 

73. (WITHDRAWN) The system according to claim 70, wherein said 
graphics operation comprises a overlay graphics operation. 
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74. (WITHDRAWN) The system according to claim 70, wherein said 
graphics operation comprises a text rendering operation. 

75. (WITHDRAWN) A method for generating a real-time video signal, said 
method comprising: 

receiving incoming MIDI control signals; 

storing previously generated video data comprising real-time video 
images; and 

generating a video output signal comprising selectable portions of said 
previously generated video data, wherein said video output signal is generated 
according to a video generation algorithm that associates said incoming MIDI 
control signals with particular real-time video images of said stored video data. 

76. (WITHDRAWN) The method according to claim 75, wherein said 
video generation algorithm is selected from a plurality of pre-programmed video 
generation algorithms. 

77. (WITHDRAWN) The method according to claim 76, wherein an 
incoming control signal of MIDI format is used to select a particular video 
generation algorithm from said plurality of pre-programmed video generation 
algorithms. 

78. (WITHDRAWN) The method according to claim 75, wherein said real- 
time video images comprise video stills. 

79. (WITHDRAWN) The method according to claim 75, wherein said real- 
time video images comprise video clips. 

80. (WITHDRAWN) The method according to claim 75, wherein said real- 
time video images comprise graphics animations. 

81 . (WITHDRAWN) The method according to claim 75, wherein said real- 
time video images comprise a graphic rendering created from a numerical 
dynamic simulation. 
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82. (WITHDRAWN) The method according to claim 75, wherein at least 
one graphics operation is introduced into said video output signal in response to 
said incoming MIDI control signals. 

83. (WITHDRAWN) The method according to claim 82, wherein said 
graphics operation comprises a raster graphics operation. 

84. (WITHDRAWN) The method according to claim 82, wherein said 
graphics operation comprises a vector graphics operation. 

85. (WITHDRAWN) The method according to claim 82, wherein said 
graphics operation comprises a overlay graphics operation. 

86. (WITHDRAWN) The method according to claim 82, wherein said 
graphics operation comprises a text rendering operation. 

87. (WITHDRAWN) A real-time video signal generation system, said 
system comprising: 

an input interface for receiving incoming MIDI control signals; 
a video signal interface for receiving an incoming video signal; and 
a real-time video processor for generating a video output signal 
comprising a modified version of said incoming video signal, wherein said real- 
time video processor modifies said incoming video signal according to a video 
generation algorithm controlled by said incoming MIDI control signals. 

88. (WITHDRAWN) The system according to claim 87, wherein said video 
generation algorithm is selected from a plurality of pre-programmed video 
generation algorithms. 

89. (WITHDRAWN) The system according to claim 88, wherein an 
incoming control signal of MIDI format is used to select a particular video 
generation algorithm from said plurality of pre-programmed video generation 
algorithms. 
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90. (WITHDRAWN) The system according to claim 87, wherein a video 
overlay operation is introduced into said video output signal. 

91 . (WITHDRAWN) The system according to claim 87, wherein said real- 
time video processor generates said video output signal using a video wipe 
processing operation. 

92. (WITHDRAWN) The system according to claim 87, wherein said real- 
time video processor generates said video output signal using a video fade 
processing operation. 

93. (WITHDRAWN) The system according to claim 87, wherein said real- 
time video processor generates said video output signal using a video blend 
processing operation. 

94. (WITHDRAWN) The system according to claim 87, wherein said real- 
time video processor generates said video output signal using a video 
solarization processing operation. 

95. (WITHDRAWN) The system according to claim 87, wherein said real- 
time video processor generates said video output signal using a video geometric 
warping operation. 

96. (WITHDRAWN) A method for generating a real-time video signal, said 
method comprising: 

receiving incoming MIDI control signals; 
receiving an incoming video signal; and 

generating a video output signal comprising a modified version of said 
incoming video signal using a real-time video processor, wherein said real-time 
video processor modifies said incoming video signal according to a video 
generation algorithm controlled by said incoming MIDI control signals. 



-33- 



Attorney Docket No. 2152-3020 



97. (WITHDRAWN) The method according to claim 96, wherein said 
video generation algorithm is selected from a plurality of pre-programmed video 
generation algorithms. 

98. (WITHDRAWN) The method according to claim 97, wherein an 
incoming control signal of MIDI format is used to select a particular video 
generation algorithm from said plurality of pre-programmed video generation 
algorithms. 

99. (WITHDRAWN) The method according to claim 96, wherein a video 
overlay operation is introduced into said video output signal. 

1 00. (WITHDRAWN) The method according to claim 96, wherein said real- 
time video processor generates said video output signal using a video wipe 
processing operation. 

101 . (WITHDRAWN) The method according to claim 96, wherein said real- 
time video processor generates said video output signal using a video fade 
processing operation. 

1 02. (WITHDRAWN) The method according to claim 96, wherein said real- 
time video processor generates said video output signal using a video blend 
processing operation. 

1 03. (WITHDRAWN) The method according to claim 96, wherein said real- 
time video processor generates said video output signal using a video 
solarization processing operation. 

104. (WITHDRAWN) The method according to claim 96, wherein said real- 
time video processor generates said video output signal using a video geometric 
warping operation. 

1 05. (WITHDRAWN) A video camera control system comprising: 
an input interface for receiving incoming MIDI control signals; 



-34- 



Attorney Docket No. 2152-3020 



a camera for generating an outgoing video signal, said camera comprising 
at least one video capture control function; and 

a processor for generating camera control signals according to a video 
capture control algorithm controlled by said incoming MIDI control signals, 
wherein said camera control signals selectively control each function of said at 
least one video capture control function of said camera. 

1 06. (WITHDRAWN) The system according to claim 1 05, wherein said 
video capture control algorithm is selected from a plurality of pre-programmed 
video capture control algorithms. 

1 07. (WITHDRAWN) The system according to claim 1 06, wherein an 
incoming control signal of MIDI format is used to select a particular video capture 
control algorithm from said plurality of pre-programmed video capture control 
algorithms. 

1 08. (WITHDRAWN) The system according to claim 1 05, wherein one 
function of said at least one video capture control function is a camera zoom 
function. 

109. (WITHDRAWN) The system according to claim 105, wherein one 
function of said at least one video capture control function is a camera pan 
function. 

1 1 0. (WITHDRAWN) The system according to claim 1 05, wherein one 
function of said at least one video capture control function is a camera tilt 
function. 

111. (WITHDRAWN) The system according to claim 1 05, wherein one 
function of said at least one video capture control function is a camera focus 
function. 

1 1 2. (WITHDRAWN) A method for controlling video camera system 
comprising: 
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receiving incoming MIDI control signals; 

generating an outgoing video signal using a camera comprising at least 
one video capture control function; and 

generating camera control signals according to a video capture control 
algorithm controlled by said incoming MIDI control signals, wherein said camera 
control signals selectively control each function of said at least one video capture 
control function of said camera. 

1 1 3. (WITHDRAWN) method according to claim 1 1 2, wherein said video 
capture control algorithm is selected from a plurality of pre-programmed video 
capture control algorithms. 

1 1 4. (WITHDRAWN) The method according to claim 1 1 3, wherein an 
incoming control signal of MIDI format is used to select a particular video capture 
control algorithm from said plurality of pre-programmed video capture control 
algorithms. 

115. (WITHDRAWN) The method according to claim 112, wherein one 
function of said at least one video capture control function is a camera zoom 
function. 

116. (WITHDRAWN) The method according to claim 112, wherein one 
function of said at least one video capture control function is a camera pan 
function. 

117. (WITHDRAWN) The method according to claim 112, wherein one 
function of said at least one video capture control function is a camera tilt 
function. 

118. (WITHDRAWN) The method according to claim 112, wherein one 
function of said at least one video capture control function is a camera focus 
function. 
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EVIDENCE APPENDIX 



Appellant relies upon Table 1 of the Summary of MIDI Messages (MIDI 
Manufacturers Association) for description of the MIDI messages and format. 
Table 1 is of record in the appellant's response to the Final Office Action. Table 1 
is reproduced on the following pages. 
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fflin I Manufacturers 
IlllUli 



I Association 



Table 1 of 4 - Summary of MIDI Messages 



APPENDIX A 



Supporting the Arts Through Technology 



The following table lists many of the major MIDI messages in numerical (binary) order. This table 
is Intended as an overview of MIDI, and Is by no means complete. 

See also: 

• Table 2 - Expanded Messages List (Status Bytes) 

• TaJileJ - Summary of Control Change Messages (Data Bytes) 

• Table 4 - Defined Universal System Exclusive Messages 

Additional messages are listed in the printed documentation available from the MMA. 

WARNING] Details about implementing these messages can dramatically impact 
compatibility with other products. We strongly recommend consulting the official 

C omplete MIDI l.Q Detailed Specificati on for additional information. 

Table L s MIDI 1.0 Specification Message Summary 
Updated 1995 By the MIDI Manufacturers Association 



Status 
07 D 0 



Data Byte<s) 
07 --no 



Description 



Channel Voice Messages [nnnn - 0-15 (MIDI Channel Number 1-16)] 



looormnn 



lOOlnnnn 



Qkkkkkkk 
Owwvw 



Okkkkkkk 
Owvww 



Note Off event. 
This message is sent when a 
note is released (ended) . 
(kkkkkkk) is the key (note) number, 
(vwvvvvi in the velocity. 

Note Cn event. 

This message is sent when a 

note is depressed (start) . 

(kxkxxxx) is the key (noes) number. 

{vwvwv) is the velocity. 



llOOnnnn 



Okkkkkkk 
Owwvw 



Occccccc 
C vwvwv 



Oppppppp 



Polyphonic Key pressure ;Af tertouch) . 
This massage is most often sent by 
pressing down on the key after it 
"bottoms out" . 

Ikkkkkkh) is the koy (note) number, 
(vwvwv) is the pressure value. 

Control Change, 

This message ie sent when a controller 
value changes. Controllers include devices 
»uch as peda) s and levers . 
Controller numbers 12 0-127 are reserved 
as "Channel Mode MessageB" (below) . 
(ccccccc) is the controller number (0-119) . 
(vwvwv) io the controller value (0-127). 

Program Change. 

This message, sent when the p*tch number changes. 
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(ppppppp) is the new program number. 



APPENDIX A 



llOlnnnn Owwvw Channel Pressure (After -touch) . 

This message is most often sent by pressing down 
on the key after it "bottoms out". This message 
io different from polyphonic after-touch . Use 
this message to send the single greatest 
pressure value {of all the current depressed keys) . 
(wwwv) is the pressure value. 

liiQnnnn oiiiilll Pitch wheel Change- 

Omrnnnnmmm This message is sent to indicate a cliange in the 

pitch wheel. The pitch wheel is measured by a 
fourteen bit value. Center (no pitch change) i3 
2000H, Sensitivity is a function of the 
transmitter. 

(11 1111) are the leaet significant 7 bits, 
( mummium i) are the most significant 7 bite. 



Channel Mode Messages (See also Control Change, above) 



IQllnnnn Occccccc Ciiannel Mode Messages. 

ovwww This the same cods as the Control 

Change (above) , but i:npl events Mode 
control and special message by using 
reserved controller nutnbere 120-127. 
The commands are* 



All Sound off. 

When All Sauivi Off it* received 
all oscillators will turn off, and 
their volume envelopes are set to 
aero ac soon as possible, 
c = 120 r v = 0; All Sound Off 

Reset All Controllers. 

When Reset All Controller© is received, 
all controller valuee are reaet to thair 
default values. (See specific Rsccmimended 
Practices for defaults) . 

C ■ 121, v ■ x: Value must only be zero 
unless otherwise allowed 
in a specific Recommended 
Practice. 

Local control. 

When Local Control ia OCC, all device,** 
on a given channel will respond only to 
data received over MIDI. Played data, etc, 
will be ignored. Local Control On 
restores the functions of the nnraal 
controllers. 

c = 122, v = 0: Local Control Off 
c = 122, v = 127: Local Control On 



All Notes Off. 

When an All Notes Off is received, 
all oscillators will turn off, 
c = 123, v = 0: All Notes Off 
(See text for description of actual 
mode commands . ) 

c = 124, v - 0: Omni Mode Off 

c - 125, v = 0: Omni Mode On 

c - 126, v = m: Mono Mode on tfdy off) 
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where M is the number of channels ADDPMnlY A 
[Omni Off) or o (Omni On) MrrtlMUIA M 

C = 127, v = 0: Poly Mode On (Mono Off) 
{Note: These four messages also cause 
All Notes Off) 



System Common Messages 



1111000Q QiiiLLii 
Oddddddd 



Odddrtddd 
11110111 



111100O1 

11110010 01111111 



11.110011 Osssssss 

11110100 
t.11101.01 
11110110 

11110111 



Systetti Exclusive. 

This message makes up for all that MIDI 
doesn't support. (iiiiiii) is usually a 
seven-bit Manufacturer's I.D. code. If the 
synthesizer recognizee the I.D. code as 
its own, it will listen to the rest of 
the message (ddddddd) . Otherwise/ the 
message will be ignored. System Exclusive 
ic used to send bulk dumps cuch as patch 
parameters and other non-spec data. 
(Note: Real-Time messages ONLY may bfi 
interleaved with a system exclusive . ) 
This message also is used for extensions 
called Universal Exclusive Messages. 

Undefined. (Reserved) 

song Position Pointer. 

This is an internal 14 bit register that 
holds the number of MIDI beatB <1 beat= 
six MIDI clocke) since tho start of 
the song. 1 is the L.SB, * the MSB. 

Song Select. 

The Song Select specifies which sequence 
or 3ong is to be played. 

Jndcfined. (Reserved) 

'Jndef i n«.d. (Rftserved) 

Tune .Request. 

Upon receiving a Tune Request, all analog 
synthesisers should tune their oscillators. 

Bnd of Exclusive. 

Used to terminate a System Exclusive 
dvimp (see aoove) . 



System ^eal-Time Messages 
11111300 

11111901 
1.1111.010 



11111011 



Timing ClocX. 

Sent 24 times per quarter note when 
synchronization is required (see text) . 

Undefined. (Reserved) 

S-arr. . 

Start -he current 3equence playing. 
(Thia ciwjBBaye will be followed with 
Timing Clocks) . 

Continue . 

Cont-lmiA at r.he point r.he saquence was 
Stopped. 
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1 1111100 

11111101 
U1111L0 



11111111 



Stop . 

stop che current sequence. 
Undefined. (Reserved) 
Active Sensing- 

Use off this message is optional. When 
initially sent, the receiver wili expect 
to receive another Active Sensing message 
each 300ms (max) , or it will be asBune 
that the connection has been terminated. 
At termination, the receiver will turn off 
all voices and return to normal (non- 
active sensing) operation, 

Reaet . 

Reset all receivers in the system to 
power-up status. This shouid he used 
sparingly, preferably under manna*! 
control. In particular, it should not 
be eent on power -up. 
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All materials, graphics, and cext copyright © 1995-2008 MIDI ManufactLrers Association Incorporated. 
Lcs Angeles, California. Use Is prohibited without written permission. 
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RELATED PROCEEDINGS APPENDIX 

There has been no Board decision for any of the applications identified in 
the Related Appeals section of this Appeal Brief. 
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